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Introduction 

Volume 13: Meeting the Accountability Challenge 



B Y MARIE CORA 



n this issue of Adventures in Assessment , 
practitioners and program directors 
write about their efforts to meet the 
rising federal and state demands for 
accountability. The field presently 
faces layers of challenges: How do we effec- 
tively determine adult learners' capabilities? 
How do we capture learners’ progress? 
How can we use data to inform our practice? 
How do we construct a system of accounta- 
bility that examines efforts at the program 
level? 

In an article first written for the ALRI 
Newsletter ; Steve Reuys sets the stage for us 
with an overview of the NRS and SMARTT 
reporting systems. He proceeds to outline 
three options regarding assessment for the 
field to meet accountability demands. 

I contributed an article that I wrote in my 
previous position as director of a cluster of 
volunteer-based literacy programs in 
Providence, Rhode Island. I was in the 
process of figuring out ways to fit our very 
non-traditional programming into the federal 
and state guidelines mandating certain per- 
formance and reporting around assessment 
and accountability. This was particularly chal- 
lenging in light of the fact that my program 
was staffed by non-professional volunteers. 

Ten years ago, in the very first issue of 
Adventures in Assessment , Janet Kelly wrote 
about the assessment process developed at 
the Read/Write/Now Program in Springfield. 
Janet writes again, reflecting on a decade of 
learning and reflection, and outlines her 
program’s efforts to involve adult students 
more fully in programming, and to use 




assessment to inform learning and teaching, 
while still meeting the demands of funders. 

Sally Gabb examines whether setting 
goals with adult students is conducted to 
meet the requirements of the Massachusetts 
reporting system, SMARTT, or if goal-setting 
is engaged with students as a process for 
learning. 

Chris Hebert and her colleagues at 
Quinsigamond Community College share the 
new ESL placement test they developed to 
meet the needs of placing students in SPL 
levels 7 through 10. Both teachers and adult 
learners were involved in this process. 

Two articles look at how data collection 
can inform and improve program practice, 
while still satisfying funders and other stake- 
holders. Diane Rosenthal gives us an 
overview of the What Works Literacy 
Partnership (WWLP), a nationwide program 
that brings together 12 adult education 
programs interested in building their capaci- 
ty to collect, manage and analyze data. Carol 
Gabler and Heidi Fisher provide a case study 
of their program’s participation in WWLP. 

The questions raised in the first paragraph 
of this introduction are questions that I have 
heard practitioners voice across the state. And 
as they are asking these tough questions, they 
are engaging in research, reflection, question- 
ing, and innovation. It's a challenging, but 
exciting time in our field: we must rise and 
meet that challenge. We’re ready. 

Your thoughts and ideas are welcomed 
and encouraged. If you would like to submit 
an article or have comments, please feel free 
to contact me at mcora@worlded.org . 



ADVENTURES IN ASSESSMENTS 



PAGE 1 



VOLUME 13 



Table of Contents 



Introduction: Volume 13: Meeting the Accountability Challenge page 1 

Marie Cora, Editor, Adventures in Assessment, SABES Central Resource Center, 
World Education, Boston, MA 

New Accountability Rules Pose Dilemma for Programs page 5 

Steve Reuys, ALRI/SABES Greater Boston Regional Support Center, Boston, MA 

Layers, Brushes, and Multi-Lane Highways: 

Examining Accountability in a Non-Traditional Program page 9 

Marie Cora, Swearer Center for Public Service, Brown University, Providence, Rl 

The Adventure Continues page 13 

Janet Kelly, Read/Write/Now Program, Springfield, MA 



Authentic Goal Setting with ABE Learners: 

Accountability for Programs or Process for Learning page 17 

Sally Gabb, SABES Southeastern Regional Support Center, Fall River, MA 

Quinsigamond Community College's Site-Specific Assessment page 25 



Chris Hebert, Anne Burke, Linda Gosselin, Arpi Hedeshian, 
Adult Community Learning Center, Worcester, MA 



What Works Literacy Partnership: Making Data Work for You page 47 

Diane Rosenthal, Literacy Partners and WWLP, New York, NY 

Analyzing Your Organization's Data To Tell Your Story page 51 



Carol L Gabler, Heidi L. Fisher, Literacy Volunteers of America - Chippewa 
Valley, Eau Claire, Wl 




ADVENTURES IN ASSESSMENT 



PAGE 3 



VOLUME 13 



5 



New Accountability Rules Pose Dilemma for Programs 



A version of this article first appeared in the July 2000 ALRl Newsletter. 



a dult basic education programs 
in Massachusetts may be 
faced with some difficult choic- 
es these days as they work to 
comply with the requirements of the new 
National Reporting System (NRS) and the 
state’s SMARTT data management system. 
One of these choices involves the assess- 
ment and reporting of individual student 
progress in literacy and language learning. 
(Other choices revolve around different 
issues, such as the reporting of individual 
student goals, but this article will focus on 
the reporting of students’ academic 
progress.) 

As of July 1 [1999], the NRS now requires 
that each state and hence each federally- 
funded ABE/ESOL/GED program report the 
progress of its adult learners in measurable, 
quantifiable terms, using two “ladders” of 
six levels each, one built for ESOL and one 
for literacy/ABE/GED. 

In Massachusetts, the state Department 
of Education (DOE) anticipated this require- 
ment by building into its SMARTT data 
management system the requirement that 
programs report student progress for all 
students. On the ESOL side, this means 
assessing students in terms of the six 
Student Performance Levels (SPL) adopted 
by the NRS for its own reporting. On the 
ABE side, DOE is requiring that programs 
report in terms of grade level equivalents 
(GLEs) from 1 to 12, which DOE will then 
translate into the six NRS levels for state 
reporting purposes. Programs are not 
required to use standardized tests to arrive 



at these SPLs or GLEs, but if they elect to 
use an alternative measure, they must cor- 
relate the results of this alternative meas- 
ure with the SPL or GLE ladder and, even- 
tually, provide proof of the validity and 
reliability of these correlations. 

These student assessment require- 
ments, as mandated by the NRS and 
implemented by SMARTT, can present pro- 
grams with some difficult choices in how 
to conduct their assessment processes so 
as to meet two goals that are at least 
somewhat in conflict: 1) meeting these 
new reporting requirements; and 2) pro- 
viding teachers and students with assess- 
ment information that is meaningful, accu- 
rate, and useful. This article will review the 
three basic options that it seems adult 
basic education programs now have 
regarding assessment. 

The first of these options is for a pro- 
gram simply to use standardized tests for 
virtually all of its student assessment. The 
basic advantage of this approach, as 
everyone knows, is that it is rather easy to 
do 6 not an insignificant reason. I would 
argue, though, it also carries a number of 
serious disadvantages. The first is that 
standardized tests simply do not appear 
to be very good ways of assessing the 
reading, writing, and math abilities of stu- 
dents, and especially of adult learners. The 
literature on this is vast and I won’t go 
into the specifics here, other than to point 
to the many articles and books written by 
Susan Lytle, Marcie Wolfe, Marilyn 
Gillespie, Elsa Auerbach, Peter Johnston, 
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"These student 
assessment 
requirements... 
can present 
programs with some 
difficult choices 
in how to conduct 
their assessment 
process." 
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and many others over the past two 
decades or more, criticizing standardized 
methods of assessing learning and pro- 
moting various types of alternative assess- 
ment. (Local references would include the 
Fall 1988 issue of Focus on Basics, and the 
numerous volumes of Adventures in 
Assessment published by SABES/World 
Education. The A.L.R.I. has many resources 
and lists of resources on alternative 
assessment, for those who are interested.) 

A second disadvantage, which could at 
least partly derive from the first, is that 
standardized tests may do a very poor job 
of capturing and reflecting the learning 
that goes on in adult basic education 
classes. In a recent posting to the NLA 
(National Literacy Advocacy) electronic list, 
Thomas Sticht discusses a new study by 
janet K. Sheehan-Holt and M. Cecil Smith, 
which finds little improvement in scores on 
the NALS (National Adult Literacy Survey) 
test by adults participating in ABE classes. 
It may thus prove to be a major risk for 
adult basic education programs across the 
country and for the system as a whole to 
be judged largely on the basis of students’ 
improvement in scores on tests that may 
be inherently incapable of capturing much 
of the learning that is taking place for 
these students at these programs. 

The third disadvantage is that, despite 
the literally hundreds of tests that have 
been produced in this country, very few of 
these are developed specifically for use 
with adult learners, and there may be cer- 
tain portions of our adult learner popula- 
tion for whom no test is appropriate. For 
example, ESOL teachers have indicated 
that the BEST test, which is used almost 
universally for determining SPL levels with 
non-native-English speakers, was originally 
developed for use with certain refugee 
populations and is not necessarily appro- 



priate for some other ESOL populations, 
especially students at higher levels. 

A fourth disadvantage is that all 
assessments must be rendered in terms of 
either SPLs (for ESOL) or GLEs (for ABE). I 
can’t really speak to how well the SPL lad- 
der works to reflect students’ English lan- 
guage achievement. However, the use of 
GLEs to report ABE progress is certainly 
problematic, though it may be mechanical- 
ly easy enough to do. Quoting briefly from 
a few sources: 

• “Problems with grade level completion 

criteria for literacy statistics are well 
documented (e.g. Coles, 1976).” 
(Hannah Arlene Fingeret, Adult Literacy 
Education: Current and Future 

Directions, ERIC, 1984, p.8). 

• “Although the problems with grade lev- 
els as indicators of adult performance 
and progress are well-established, their 
use in the field of adult literacy is sur- 
prisingly pervasive.” (Susan Lytle, 
Thomas Marmor, and Faith Penner, 
paper presented in 1986). 

• “Critics of the use of grade levels, 
however, point out that there is no 
valid translation indicating what real 
world literacy skills correspond to com- 
pletion of a certain number of years in 
school.” (Carolyn Chase Ehringhaus, in 
the Adult Education Quarterly, 1990, 
Vol. 40, No. 4, p. 189). 

• “Test results that give grade level 
scores or indicate that learners can 
identify specific skills on paper-and- 
pencil tasks yield very limited informa- 
tion. Despite the fact that our society in 
general seems quite impressed with 
measurable results that can be report- 
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ed numerically, such data fail to match 
the overall goals. The assumption that 
numerical scores give evidence of con- 
fidence and competence is highly ques- 
tionable.” (Rena Soifer, et al, The 
Complete Theory to Practice Handbook 
of Adult Literacy, Teachers College 
Press, 1990, p. 171) 

So, while using standardized tests as 
the sole means of assessment may be rel- 
atively easy, there are numerous other dif- 
ficulties and risks associated with that 
route. A second possible assessment 
option for programs is to use various 
means of alternative assessment and to 
translate the results of these assessments 
into GLEs and SPLs. The major advantage 
to this approach is a very important one: it 
would provide assessment information 
that creates a much fuller picture of a stu- 
dent’s literacy abilities that is likely to be 
more meaningful and much more useful to 
teachers and students alike. 

There are again, however, several likely 
disadvantages as well. The first is the time 
and energy it would take to create or 
adapt these methods of alternative assess- 
ment for use at a particular program with 
a particular population of students. It 
should be noted, though, that a great deal 
of work has already been done in this area 
(see, for example, the various Adventures 
in Assessment volumes) and more could 
be supported by targetted funding from 
the state Department of Education. 
Secondly, there will Se the difficulty of 
proving to a sufficient degree the validity 
and reliability of these measures, though 
obviously the criteria set for achieving this 
level of proof will in large part determine 
how difficult this task will be for individual 
programs. Again, this difficulty could be 
mitigated through collaboration on the 



part of various programs and the support 
of DOE funding. 

A third disadvantage is found in the 
requirement that these alternative assess- 
ments must be translated into SPLs or 
GLEs. Alternative assessment is not merely 
another way of getting to the same place; 
it is also to some degree a different desti- 
nation. Alternative assessment is based on 
a view of literacy and learning that doesn’t 
see learning to read and write and do 
math as activities that can be laid out in a 
neat, sequential series of skills through 
which all learners progress from bottom to 
top. Alternative assessment approaches 
attempt to create a picture of a learning 
process that is by its very nature non-lin- 
ear and that can vary tremendously from 
person to person. Having to translate, at 
least on the ABE side, alternative measures 
of assessment into GLEs certainly acts to 
negate the original intent and meaning 
and value of the alternative assessment 
process. 

A third option which programs have is 
to combine elements of the first two 
(including their advantages and disadvan- 
tages) by using both standardized tests 
and alternative assessments. This hybrid 
option would use standardized test results 
to meet the requirements of the new 
reporting system in a relatively easy way, 
while using an alternative assessment 
approach to provide meaningful and useful 
information to teachers and students. This 
option would still require programs’ time 
and effort to develop alternative assess- 
ments and would still run the risk of not 
capturing for reporting purposes the actu- 
al learning that is going on in classes. 
Nevertheless, this option may be the best 
of those available. 

In the long run, we as a field will need 
to “assess” how well the new approaches 
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to assessment and accountability— the 
NRS and SMARTT systems— are capturing 
and reflecting the learning that students 
achieve as they attend our classes. 



Steve Reuys is Staff Development 
Coordinator at the A.L.R.I./SABES Greater 
Boston Regional Support Center. 
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Layers, Brushes, and Multi 
Examining Accountability in a 
Non-Tradit ional Program 



O ur work at the Swearer Center 
is pretty messy stuff. We 
develop non-traditional mod- 
els of education that meet 
needs not met by traditional sys- 
tems already in place. Much of this work 
involves tailoring programming to individ- 
uals. Having worked in education for 16 
years, the biggest lesson I have learned is 
that there is nothing as individualistic as a 
person’s education. Swearer Center pro- 
grams strive to help people identify what 
works for them in the learning process, 
and then to help them acquire the 
resources necessary for supporting their 
efforts. For this methodology to be suc- 
cessful, it means that the many players 
involved must collaborate carefully and 
consistently. We must be able to count on 
each other. 

The Programs 

The language and literacy programs 
that I supervise are focused on helping 
people gain basic skills they need, for 
example, to find a job or a better living 
wage, help their children with schoolwork, 
or conquer ESL at the high school level. 
Presently, there are seven programs that 
follow this focus on three ABE programs 
(Adult Basic Education) and four ESOL pro- 
grams (English to/for Speakers of Other 
Languages). Each program is supervised by 
a part-time, paid program coordinator and 
staffed by volunteer tutors. All coordina- 
tors and volunteers are college students. 



the majority of whom are studying at 
Brown University. The language and litera- 
cy programs receive funding from the 
Rhode Island Department of Education 
(RIDE) in addition to being supported by 
the Swearer Center. All the services we pro- 
vide to the community are free of charge. 
This web of individuals presents significant 
resources brought to bear on the literacy 
needs of Providence communities, with 
student leaders at the helm. 

Molly, a sophomore concentrating in 
History and Latin American Studies, coor- 
dinates Partners in Education (PIE), an 
adult basic education program which 
works exclusively with teenage mothers. 
PIE pairs female Brown students with 
female community members who have 
children and have left high school before 
graduating. PIE provides individualized 
educational programming at learners’ 
homes, making schooling accessible. 
Presently, PIE maintains twelve learning 
partnerships that meet for approximately 
three hours per week. 

Allison, a senior concentrating in 
English, coordinates the Pawtucket ESOL 
Program, which offers small group instruc- 
tion four nights per week to immigrant 
community members from an area just 
north of Providence. Three levels of 
instruction are offered: ESOL Literacy, for 
individuals with no literacy skills in any 
language, and Beginning and Intermediate 
ESOL. Volunteers team-teach classes until 
they are comfortable enough to lead a 
class on their own. Presently, the 
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Lane Highways: 

BY MARIE CORA 

This article was written 
in spring of 2000 as 
I prepared to leave my 
directorship of Swearer 
Center for Public Service 
programs after nine 
years. The Swearer 
Center for Public 
Service is a non-academic 
department of Brown 
University whose 
mission is to engage 
undergraduate students 
and community members 
in examining issues 
of social justice, and 
to develop collaborative 
programming responsive 
to the needs of the greater 
Providence community. 
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Pawtucket program engages 12 volunteers 
and 40 adult learners. 

The Layers 

Neither the program coordinators nor 
the volunteers necessarily have a back- 
ground in literacy or teaching. It is my 
responsibility to provide leadership and 
support for them so that they in turn can 
effectively provide educational opportuni- 
ties for community members. The pro- 
grams’ structure and methodology stem 
directly from the learners’ needs and 
strengths. The college students, while new 
to the craft of teaching, also offer their 
optimism, energy and creativity to the 
community, making learner-centered, pro- 
gressive classrooms a possibility. 

In my role as the supervisor of these 
programs, I am responsible to our learners 
and the new skills and knowledge they 
seek; to program coordinators and volun- 
teers and the guidance and training they 
need; to funders who seek concrete 
results; and to my colleagues who trust my 
leadership and rely on my expertise. Each 
of these roles requires different behavior, 
involves different expectations of me, and 
is driven by different assumptions of the 
roles I play. As with many other areas of 
work, responsibility and accountability 
come in layers. And because we are an 
educational institution, the learning comes 
in layers as well. As I said, this is messy 
work. 

Brushes with Accountability 

As I enter my ninth year in this job, I 
realize that my expectations and assump- 
tions have changed drastically since 1992. 

I entered this job fiercely believing (and 
still believe) that effective teachers must 



necessarily have formal, rigorous training, 
and without it, they cannot possibly be 
held accountable for their work. How can 
someone who has never digested theory 
and tested their practice be an effective 
educator? I entered this job somewhat 
skeptical that youthful volunteers could 
carry out work that I had spent six years 
attempting professionally. This was my first 
brush with accountability at the Swearer 
Center: I took the job, so it was my 
responsibility to figure out how to educate 
and support a novice group in this vital 
endeavor. 

I lost a little bit of weight and a lot of 
sleep those first years. I thought my role 
was to coordinate and support teachers in 
their effort to bring literacy services to 
members of the community. But my role is 
actually to develop those teachers while 
they struggle to help other people learn. I 
thought I would be working with teachers 
of a certain caliber, but I work with stu- 
dent-teachers who are striving to build 
their teaching skills. While I had to change 
my assumptions about who I would be 
working with, I also learned that I was now 
among a real community of learners. In our 
programs both the “students” and the 
“teachers” were clearly learning new skills. 

My second brush with accountability 
involved one of our funders, the Rhode 
Island Department of Education (RIDE). As 
the director of these literacy programs, I 
am responsible for making sure that RIDE 
receives documentation which indicates 
that we are indeed conducting our literacy 
business as outlined in the grant guide- 
lines. Often in adult education this is our 
first responsibility; after all, without fund- 
ing, we won’t exist. 

I have slowly shifted my perspective 
about accountability to the funder. The 
grants that I write are practical, carefully 
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crafted, and feasible. Over the years my 
student staff and I have done our best to 
fit into what the grant states. But I have 
come to believe that the funder is the least 
important player in this game, even though 
we depend on that money. Regardless of 
new guidelines or requirements or federal 
programs, I must remain accountable first 
to the adult learners who attend our pro- 
grams. 

Expectation and Assumption 

In the cascading structure of our pro- 
gramming the question of who is account- 
able to whom and for what is significant; 
embedded within are each person’s expec- 
tations and assumptions. In thinking about 
what the adult learner needs from us, I 
find myself working backwards: if the 
learners needs x, then the volunteers need 
y, which means the program coordinators 
need z. I can only make myself account- 
able to the learner through the work I do 
with the program coordinator. It is crucial 
then, that I fully understand what each 
constituent needs and how he or she 
defines expectations of one’s self and of 
those with whom he or she is working. 
Those expectations must be the same. The 
content and the process might look differ- 
ent depending on the individual and their 
role, but each person must be challenged, 
must be presented with relevant material, 
and must be given opportunities commen- 
surate with his or her capabilities. 

In a staff meeting with my program 
coordinators we were discussing uses of 
creativity in language teaching. It emerged 
that many volunteers had been using 
games such as “Simon Says” and “Head, 
Shoulders, Knees and Toes” in their class- 
es in units on the body. This is a viable 
technique, but one that must be used with 



other methods to connect new vocabulary 
with experiences that are a part of learn- 
ers’ lives. In other words, this vocabulary 
must be put into a context that is relevant 
to the learner. Otherwise, “heads, knees, 
and toes” are words in a song that is 
remembered for only a short time. 

In further discussion, it emerged that 
many volunteers were having difficulty pro- 
viding real and practical context when 
teaching new vocabulary. I asked, “What 
methods and activities do YOU find to be 
most helpful when learning new vocabu- 
lary in another language?” and the 
response took me by surprise. None of my 
program coordinators felt that using game 
techniques would be effective for their own 
language learning. Why, then, I asked, 
should they assume that adult learners 
would find this method helpful or appro- 
priate? Shouldn’t the adult learner studying 
the names of body parts be equally as 
engaged as the Brown student studying 
microbiology? Shouldn’t the adult learner 
be presented with material relevant to her 
life (“How will you describe to your doctor 
the kinds of pains that you feel in your 
chest?”) just as the Brown student should 
be presented with the practice of microbi- 
ology in the 21st, not the 17th century? 

Accountability is a Multi-Lane 
Highway 

What does it mean to count on some- 
one? What does it mean to be counted on? 
With the layers of staff we have at the 
Swearer Center, accountability becomes not 
a two-way street, but a multi-lane highway. 
The adult learner is counting on her volun- 
teer tutors to teach her what she wants to 
know; the tutors are counting on their pro- 
gram coordinator to help them figure out 
how best to help people learn; the coordi- 
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To be effective, 
we need to identify 
and articulate our 
expectations of one 
another, utilize 
our assumptions 
in productive ways, 
and develop methods 
for accountability 



nators are counting on me to help them be 
effective administrators and teachers for 
their volunteers. 

These layers of trust demand that we 
continuously examine our assumptions 
and expectations. To be effective, we need 
to identify and articulate our expectations 
of one another, utilize our assumptions in 
productive ways, and develop methods for 
accountability for all participants. If we do 
this, then we are ahead of those grant 
guidelines: the grant can then reflect the 
work, not vice versa. With a structure of 
accountability and trust in place, we create 
room for creativity and growth for all of us. 
We take advantage of the messiness to 
explore new and effective ways to learn and 
teach together. 



for all participants. 

Marie Cora is now a Staff Development 
Specialist for SABES/World Education , 
and the editor of Adventures in 
Assessment. 
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The Adventure Continues... 



It has been about ten years since I 
first wrote a piece for Adventures in 
Assessment. Everything has changed 
and nothing has changed at 
Read/Write/Now over the past decade. 

The reasons for improving the tools 
and practices of assessment of learning 
and teaching are still the same. 
Meaningful, effective assessment helps 
learners on the sometimes long and wind- 
ing road of adult basic education know 
they are getting somewhere. It also gives 
learners and teachers important informa- 
tion about the next steps to take to reach 
their goals, great and small. If any of us 
are to be “lifelong learners,” it’s essential 
that we not only keep learning new things, 
but know why we are learning them and 
what to do with new knowledge and skills. 
The more involved we are in directing and 
assessing our own learning, the more like- 
ly we are to be able to take what we learn 
in one setting, and use it and build on it 
in another. 

If this applies to adults who have had 
a reasonable chance at education and 
have adequate literacy skills, it applies 
even more so to adults who have not yet 
had success in education or an opportuni- 
ty to get one. If I had started graduate 
school with the same unanswered ques- 
tions that many learners have as they start 
in adult basic education programs, (When 
will l be done? Whot does it mean to be 
done? What will / be able to do at the end 
of it that i can't do now?), I probably 
would never have started. It is the respon- 



sibility of programs like ours to work with 
learners to answer those questions in rela- 
tion to each individual learner as well as 
to the program as a whole. 

Assessment that works is also impor- 
tant for teachers as they work to connect 
learners’ goals, needs, and interests with 
curriculum and learning activities. In a per- 
fect world, goal setting, curriculum, and 
assessment are all linked together. In this 
world, we keep working to make those 
pieces connect. 

Ten years ago, the assessment tools we 
developed were mostly used by teachers 
to document learners’ progress and share 
with learners on a regular basis. The stu- 
dent portfolios then were more like 
teacher-directed collections of learners’ 
work. 

Learners’ portfolios are now their own. 
There is class time scheduled to introduce 
them, to put them together in loose-leaf 
binders that are kept on shelves in each 
meeting space for classes, and to choose 
things to include in portfolios. Learners 
write something about why they chose the 
item and what it means to them in terms 
of progress toward a goal. 

We still do lots of goal setting in a 
variety of ways with a variety of tools, 
including a revised version of the 
Reading/Writing Goals List we used ten 
years ago. Learners’ goals and interests 
help form the curriculum in each group. 
Learners choose a limited number of goals 
to focus on each session and teachers 
have goal setting and goal review confer* 
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ences three times per program year with 
each of their reading & writing and math 
students. Learners keep track of what they 
are doing and learning with writing 
records, book lists, and math activity 
records. 

Many learners reftect on their reading in 
Reading Response Journals and all use 
Dialogue Journals. Published writing, a 
book review, a resume, a research project, 
a copy of a driverfs license earned, and 
math work might all find their way into a 
portfolio. Three times each program year, 
we reassess reading progress with a com- 
bination of an adapted version of the New 
Readers Press Whole Language for Adults 
Reading Inventory. We have added read- 
ings using the Fleisch readability scale to 
get grade levels and we have added recall, 
interpretive, and active questions with a 
scoring scale to make the assessment less 
subjective. This is not the only way read- 
ing is assessed, but it is the way that is 
used for marking progress within classes 
and moving people to other classes within 
the program. 

Reading Miscue Analysis is a powerful 
tool of assessment we are still trying to 
incorporate into program practice. Some of 
us who have been working at 
Read/Write/Now for five or more years have 
made multiple stabs at doing Miscue 
Analysis with learners. Over the past year, 
we have had several in-service training 
sessions with a consultant from UMASS, 
Dr. Patricia Silver, and teachers are doing 
Miscue Analysis with some of their devel- 
oping and intermediate students. It is very 
useful in analyzing the strengths and 
needs of readers and planning instruction, 
but it takes time and practice on the 
teacher's part, as well as individual time 
with a learner to tape oral reading, and 
then to meet with them after the analysis 



is done to share it. We are committed to 
making it part of our assessment practice, 
but it is far from institutionalized yet. 

One of the things about the world of 
adult basic education that has changed in 
the past ten years is the degree of 
accountability required by most funding 
sources. Funders want to know much more 
about many more things now. 
Unfortunately, they all have their own spe- 
cial way for you to demonstrate that your 
program is doing what they are giving you 
money to do. If a program has multiple 
funders, as we do, there may be require- 
ments that mean collecting anecdotes, 
writing long narratives, doing case studies, 
collecting detailed data on every learner 
while they are in the program and finding 
out about their lives after they leave, using 
pre-tests, post-tests, reporting on goals set 
and met, reading levels attained, and 
attendance. It seems reasonable that when 
money, public or private, is invested in a 
program to accomplish certain things, the 
program must be accountable for accom- 
plishing those things. It does not seem 
reasonable that being held accountable 
often means devoting more hours to ful- 
filling the funders' ideas of accountability 
than to doing useful assessment with 
learners, who are the people that all the 
funding is supposed to be serving. In that 
perfect world, the tools that work for a 
program would be acceptable to its fund- 
ing sources too. 

Regardless of the world of grants and 
reporting requirements, the essential pur- 
pose of assessment is informing learners 
and teachers of where they are on the road 
to wherever the learner wants to go, and 
helping them to decide on the next steps 
or new directions to take. The challenge 
has grown from finding effective tools and 
processes for meaningful assessment to 
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doing it while meeting the requirements of 
funders without duplicating efforts or cre- 
ating dual systems of assessment. 

So far, we have not found a satisfacto- 
ry way to avoid the duplication of efforts 
and duality. One set of goals lists and 
assessment measurements is the “real” 
one in terms of what learners and teachers 
use to understand progress and the other 
is the “real” one in terms of what must be 
reported to funders. Neither one seems to 
fully capture the growth and progress we 
see happening in the lives of adult learn- 
ers. The process of continually reviewing 
and revising tools and procedures for 
assessment has never stopped, but the 
sheer volume of new requirements, man- 
dates, technological changes, and informa- 
tion of every kind that has characterized 
our culture over the past decade has creat- 
ed its own kind of inertia. Add to the mix 
that funding for literacy is still insecure and 
insufficient, making continuity in staffing 
impossible. 

It’s too easy to fall into a kind of pas- 
sive/reactive role regarding assessment 
and accountability to funders. I agree with 
much of what Heide Spruck Wrigley wrote 
in her 1998 article, “Assessment and 
Accountability: A Modest Proposal”. If we 
don’t try to develop meaningful frame- 
works for teaching and assessment that 
truly reflect our practice and the kinds of 
successes learners achieve in our pro- 
grams, Those Who Must Be Reported To 
will fill the void. This has already hap- 
pened in many ways, but there is always 
hope and the politics of education — just 
like all politics — are subject to change 
initiated by human beings on many levels. 
At best, we can hope to influence the pol- 
icy-makers with the stunning validity of our 
assessment instruments and what they 
demonstrate, and at worst we can fail to 



influence Those Who Must Be Reported To, 
but still develop more clarity and purpose 
within our programs and offer learners and 
teachers a more understandable path 
defined by real markers of progress that 
reflect skills learned and used and goals 
met. In Massachusetts, it seems the door 
is still open on this process. Maybe it truly 
is open and maybe it just seems open, but 
being guardedly optimistic, I say we may 
as well assume that we still have a voice 
and use it. 

At Read/Write/Now, we've been work- 
ing, sporadically, on our own teaching & 
assessment frameworks for reading & writ- 
ing for almost a year. We consulted with 
Jane MacKillop, editor of Whole Language 
for Adults, a set of resources published by 
New Readers Press that we have found 
useful. We started with the intention of 
correlating our framework with the six lev- 
els described by the National Reporting 
System, but decided along the way that we 
needed to have something that made 
sense to us; we would worry later about 
translating it into NRS for reporting pur- 
poses. 

We did look at the NRS levels and try 
to make connections with it. We also tried 
to link our framework with the 
Massachusetts ABE English Language Arts 
Curriculum Framework, which was not hard 
to do. Jane MacKillop led us through a 
process of looking at learners’ writings, 
describing what we saw evidence of, and 
deciding through this process what quali- 
ties are common to beginning, developing, 
intermediate and GED writers in our pro- 
gram. She facilitated a similar process with 
reading, through which we named some 
entry and exit texts as well as the qualities 
of texts at various levels and what readers 
do at different stages of reading develop- 
ment. The next stage involves each teacher 
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taking the list of descriptors of writers and 
readers at the levels of their classes and 
turning them into a checklist to try with 
learners to get their take on them and see 
how relevant they are in practice. 

We are using the following questions to 
guide us in developing our framework for 
teaching and assessment. What does the 
typical reader/ writer do at each level? 
What knowledge or skills does each per- 
son bring to writing or reading? What 
skills are they developing? What skills or 
strategies need to be mastered before a 
person is ready to move on to the next 
level? What literacy experiences do they 
need in order to progress? What skills and 
strategies are being modeled or taught? 

We are still revising and refining the 
program’s reading & writing framework. 
Instead of whole books as exit texts, we 
are developing a selection of shorter read- 
ings that learners and teachers can choose 
from. The reading and writing checklist 
items will be given numerical values so 
that attaining an agreed upon number of 
skills in each level along with successfully 
reading and understanding the exit texts 
will signal a move to the next level group. 
Progress within the beginning, developing, 
and intermediate levels will also be 
marked in this way. Whether we call it a 
rubric, a framework, or a series of check- 
lists, this tool should make sense to us as 
learners and teachers, and will be used to 
make transitions between classes within 
our program smoother for learners and 
teachers. It will need to be reviewed and 
revised on a regular basis to stay current 
and useful. At least theoretically, it could 



be correlated to the NRS and used as a 
tool for external reporting. 

During the past five years, we have 
engaged in a variety of projects that have 
increased the participation by learners in 
every aspect of Read/Write/Now. Learners 
have been on the Health Team, doing 
research and social action theater, men- 
tored other learners, become Peer Tutors, 
served on program planning committees, 
been elected to serve on the program’s 
Advisory Committee, acted as editors of 
the monthly student newsletter, been 
members of the Parent Educator Project 
team, and conducted various action 
research projects. 

These projects have made our claim of 
being a “learner-centered” program more 
legitimate than it used to be, and they 
have strengthened the program with new 
energy and ideas. We still have a lot to do 
to make our assessment process really 
work well for learners and teachers, but we 
are on what feels like the right road. 

It remains to be seen if what we devel- 
op will also be meaningful to funding 
agencies. If so, we’ll rejoice and have more 
time to do interesting things that improve 
the program. If not, we’ll continue to do 
what we must to fulfill our own needs for 
assessment and the program’s funders’ 
needs for accountability. Everything 
changes and nothing changes. 



Janet Kelly has been a teacher and the 
Project Manager at Read/Write/Now since 
1987 . 
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Authentic Goal Setting with ABE Learners: 



Accountability for Programs 
or Process for Learning 



BY SALLY GABB 



a dults who enroll in adult basic 
education classes know why 
they have come back to 
school. Each states her/his 
purpose clearly: ... "to learn English, to get 
a GED, to make a better life for me and my 
family." These purposes have propelled 
adult learners, despite the stresses of fami- 
ly, community and work, to dedicate time to 
gaining needed basic skills. But for most, 
this clarity of purpose does not easily trans- 
late into crafted 'goals,' outlined as a 
metered time line with identified bench- 
marks, and ’documentation' of achievement 
towards a stated goal or goals. 

During its ten-year history, Adventures 
in Assessment has provided a rich record 
of creative and dynamic efforts by 
Massachusetts ABE practitioners to devise 
classroom and program support systems 
for learner goal-setting support that will 
enable learners to redefine their purposes, 
and to understand why and how to set 
goals. In 1994, Dulaney Alexander wrote: 

* language learning is a lengthy process 
for most of us. None of our students will 
master English before leaving our program. 
Some students will leave to work , some 

because they must move too far from 
school , some because their families can't 
afford their time for school , and some 
because they decided the " perfect " English 
is not a realistic short term goal. The skills 
that enable the student to treat learning 
English as a personal project , whether in 
school or out , begin with assessment and 



goal setting. These are probably as 
-valuable as anything else we teach.* ** 

Alexander's eloquent description of the 
importance of classroom self-assessment 
and goal setting could be applied to any 
ABE, GED or ESOL classroom, to comple- 
ment and build on the purposes which 
learners can easily identify. His description 
embraces what many consider truly learn- 
er-centered goal setting, with timelines 
and outcomes that are as mutable as life 
experience. A test or a time limit will not 
easily capture the value of such a process 
for the learner. 

Goals and Outcomes: What Are we 
Counting? 

In recent years, the Massachusetts 
Department of Education (DOE) has inte- 
grated learner goal setting and outcome 
assessment as part of the data require- 
ments for the performance accountability 
system. The system requires funded pro- 
grams to elicit primary and secondary 
goals for each learner, with performance to 
be measured during the one-year grant 
period for 'primary' goals. In part, this ele- 
ment in the Massachusetts performance 
reporting system was in response to the 
National Reporting System, which desig- 
nates and defines 'core outcomes'. While 
'educational gain' is the 'Core Outcome 
Measure' #1, others include: 2) entering 
employment, 3) retaining employment, 4) 
acquiring a secondary diploma and 5) 



“....for most (adult 
learners), this clarity 
of purpose (for attending 
ABE programs) does 
not easily translate 
into crafted goals. . . " 
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entering post secondary education or train- 
ing. The NRS requires learner goal setting 
as a measure of program performance as 
stated in the official NRS Implementation 
Guidelines. "The NRS will assess program 
performance by comparing students' out- 
comes to their stated goals.” 

According to seasoned practitioners in 
the Massachusetts system, goal setting is 
fully supported by the field in concept. 
Practitioners across the state have encour- 
aged recognition of learner-generated 
goals, and outcome measures other than 
pre-post test statistics for purely academic 
gain. But the purposes for goal setting as 
envisioned by practitioners, and the 
process through which learners should be 
supported in setting goals, differs radically 
from the procedures outlined in the NRS 
Measures and Methods. The NRS has as its 
stated goal 'to establish an accountability 
system for the federally-funded adult basic 
education programs.' 

SABES has been given the responsibili- 
ty for supporting program efforts to meet 
dual goals: 1) to encourage classroom- 
based learner-centered formative goal set- 
ting and goal attainment evaluation with 
learners, and 2) to usher in collection of 
goal setting and goal attainment data 
through the SMARTT system. A group of 
practitioners in the Southeast region of 
Massachusetts recently responded to ques- 
tions about the challenges in encouraging 
learners to set goals during the intake 
process for their participation in the ABE, 
ESOL, or GED class: 

"No matter how limited their reading skills 
are , or how limited their English is, the 
first response is always “to learn English ”, 
or “to get my GED”. Learning how to set 
goals is part of instruction . " 



"You have to get to know a learner, gain 
trust, establish dialogue before that learn- 
er will be able to articulate what he or she 
hopes to accomplish, what the ' goal 1 is. In 
the beginning, most learners don't even 
understand the question if you ask, 1 what 
are your goals in the class for this yeaf'. • 

According to Elsa Auerbach and others, 
assessment of Adult Basic Education stu- 
dent outcomes is 'authentic' only when 
based on learner identified purposes for 
attending ABE classes. 'Authentic assess- 
ment' describes goal setting and self eval- 
uation as a process, to be carried out over 
time in dialogue with the practitioner, 
measuring skills and content knowledge in 
the context of real life tasks. 

In the still-evolving SMARTT data col- 
lection system, the Adult and Community 
Learning Services division of DOE has cre- 
ated a lengthy list for programs to use as 
a pre-post measure, with a range of possi- 
ble 'goals', but the requirement that goal 
attainment be measured and verified. The 
list was drawn from both the NRS desig- 
nations and a survey of the field, from 
work with programs and their experience 
with learner-centered goal setting (see 
Adventures in Assessment , Volume 4, April 
1993). But programs and practitioners sug- 
gest that the very nature of such a pre- 
post system seems to contradict reflective 
ongoing self-assessment. The procedures 
defined by Massachusetts DOE are directly 
counter to a process that enables the 
learner to develop goals and learning 
objectives over time, and to change those 
goals and objectives as he/she grows as a 
learner. 
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Classroom Self Evaluation and 
SMARTT System Goal Setting: 

A Not So Complementary Relationship 

While the user’s manual for the 
SMARTT system also encourages programs 
to be truly 'learner centered' and 'open 
ended' in the goal setting process, most 
report difficulty in carrying out the activity 
as suggested in the manual. For some 
classes, particularly lower level monolin- 
gual (English only) ESOL classes, such 
activities are limited severely by lack of 
vocabulary and ability to express ideas in 
English. Even in ABE classes, most stu- 
dents are hard pressed in the initial inter- 
view to say more than 'to read better', or 
'to get my 6ED in the long run'. The first 
is measurable by reading tests, but such 
tests may not reflect the kind of advances 
lower level learners make in the beginning. 

The outcomes of these behavior 
changes may only be evident several years 
after the initial changes. In addition, the 
learner’s primary goal for attending classes 
may be difficult for ABE programs and 
practitioners to measure, such as 'improv- 
ing my communication on the job’, 'being 
able to help my children more with home- 
work', or even 'feeling better about myself 
as a human being because I'm coming 
back to school.' 

An Ambitious Example: Evolution and 
Devolution at One Program 

In 1993, Amesbury Even Start described 
a unique and complex goal setting and 
evaluation process for its Even Start fami- 
ly literacy program (see Adventures in 
Assessment , Volume 4, April 1993). This 
system included in depth intervewing of 
each learner, and regular self-assessment 
on the part of the parent learners. The con- 



tent was often directed to a combination 
of self-esteem issues, practical applica- 
tions of literacy, and application to par- 
ent/child interaction. 

The Goal Sheet uncovers a variety of 
assessment information, but at the same 
time acts as an extremely inviting, unin- 
timidating way to assess initial capabili- 
ties, attributes and interests. The Goal 
Sheet is a visual representation of an 
eight-week cycle. It is broken up into cat- 
egories representing program require- 
ments at Even Start, including adult edu- 
cation, parenting skills, child development 
and parent child together time. These cat- 
egories are then subdivided into short 
term and long term goals. 

The goal setting process at Amesbury 
Even Start, according to Susan Martin, 
Sandra Hall and Jeanette Bahre, provided 
initial information to case managers, 
teachers and learners "without the use of 
extensive formalized testing or frequently 
inaccurate initial self evaluations." 

The ongoing interaction around 
strengths, challenges and goals enabled 
the case manager, instructor and learner to 
work towards understanding how eight- 
week program options could lead to long 
term goals. The Amesbury authors 
stressed that effective goal setting could 
only occur when a process was in place 
that allowed learner and staff to revisit 
such affective areas as dreams and wish- 
es, and do an analysis of "felt" and "real" 
strengths and challenges. A key aspect of 
the process was the way that standardized 
testing (the CASAS) was integrated with 
the extensive interview process as a way 
to enable the learner to choose program 
components, e.g., her short-term goals. 

As noted by the authors . . . " the sys- 
tem is used in seif assessment as an adult 
learner builds [short and long term goals) 
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based on the information he/she reflects 
upon , and as the work is continuously 
defined and redefined for the future . " 

In a phone Interview* this writer learned 
that the Amesbury program no longer has 
a family literacy grant, and the well-devel- 
oped model is no longer in use at the cen- 
ter. Nevertheless, the model stands as an 
impressive attempt not only to develop a 
systematic process for enabling learners to 
develop appropriate and attainable goals 
over time, but also to learn some strate- 
gies for setting appropriate goals. 

In-Class Assessment as a Mediated 
Teacher/ Learner Dialogue: 

A Practitioner Targets Barriers 
to Persistence 

In 1994, Loretta Pardi and Estelle 
Williams of the Harborside Community 
Center worked together to develop effec- 
tive in-class self-assessment/teacher 
assessment tools. Loretta designed the 
'Record of Participation 1 that both she and 
each learner completed on a weekly basis. 
This instrument (see Adventures in 
Assessment , Volume 7, December 1994) 
allowed the learner to rate him/herself on 
a wide variety of areas, including atten- 
dance, willingness to try new work, inde- 
pendent and group work, class participa- 
tion, and accomplishment of job skills. In 
addition, it provided space for the learner 
to identify specific academic skills mas- 
tered in math, writing and social studies. 

“in the first weeks of using the record with 
my class , / found that most of the students 
tended to give themselves lower ratings 
than / gave them . This not entirely unex- 
pected result gave rise to a useful class 
discussion about low self-esteem. As 
weeks progressed , however, and the class 



became accustomed to the weekly routine 
of reflection and assessment , / found that 
we usually concurred.” 

Loretta reported that she set up confer- 
ences with students when noticeable dis- 
crepancies occurred between a student's 
self assessment and her own. This sur- 
faced issues besides actual performance, 
including fear and attachment to the class. 

In a recent follow-up phone interview 
with the Harborside Community Center, 
Loretta confirmed that she still incorpo- 
rates an ongoing goal setting process into 
her practice. Loretta explained that she 
currently focuses on enabling learners to 
establish their own 'comfort zones' as the 
most important component of goal setting. 

“I work with many learners who are get- 
ting ready to transition from E 50 L to the 
pre-GED program . I spend a lot of time 
talking about what they might expect from 
the new program. I ask them to consider 
issues such as work schedule and child- 
care, which could interfere with their abili- 
ty to learn, i ask them to consider their 
learning styles and strategies , and to con- 
sider what is realistic for them. 

For example , when students tell me they 
have a hard time remembering what they 
learned from one session to the next , / ask 
them to look at how to set short term 
achievable goals . I stress that setting new 
goals is a successful accomplishment 
when the original goal isn’t accomplished. 
/ help students learn how to break goals 
into manageable pieces.” 

Loretta noted that the whole idea of 
enabling the learner to find his/her 'com- 
fort zone', and working to achieve goals 
outside the class that will set the stage for 
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participating in class and reaching aca- 
demic goals are not, and probably can not, 
be a part of the SMARTT data. She report- 
ed that in her program, as in many others, 
the initial goal setting for reporting in the 
SMARTT data is carried out by the coun- 
selor, and that she does her own goal set- 
ting work in class. This separation between 
SMARTT system goal setting and class- 
room goal setting seems to be a common 
occurrence. But, as Loretta described, with- 
out the developmental process for goal 
setting, the recorded goals will not reflect 
a valid student purpose, or a true picture 
of learner goals attained. 

Goal Setting: In Conflict with 
Language and Culture? 

Katy Hartnett, ESOL specialist for the 
Adult Literacy Resource Institute in Boston, 
says goal setting as a pre-post summative 
system is a limiting picture of the ESOL 
student audience and the many goals they 
have for seeking education. She explained: 

“This is especially true for ESOL learners , 
who can’t express in English either short 
or long-term goals. Most ESOL programs 
settte for ' improve English’ because it’s 
the only goal most new learners can 
express , and it fits the SMARTT guidelines. 
And while it IS the primary goal for most 
ESOL learners , it doesn’t capture the other 
attending goals. The cultural journey for 
ESOL learners is complex and multi-dimen- 
sional.” 

In his 1998 action research project, 
practitioner Glenn Cotton described sever- 
al 'authentic' ways to identify learner 'pur- 
poses' for improving English. The teacher 
may show the learner with very limited 
English a group of photographs or draw- 



ings depicting people in a variety of social 
situations. Learners are asked to choose 
those that are most important to them for 
learning English. Or learners may be asked 
to draw their own pictures of situations in 
which they feel it is most important to be 
able to use English. 

Glenn noted: . . .“For a goal setting 
activity to be truly useful , especially with 
beginning level ESOL learners , it seems to 
me that it needs to be integrated into a 
learning activity which assists learners to 
understand the questions being asked of 
them , and which guides them to reflect on 
and discover what their authentic 
needs/goals are.** 

Both Katy and Glenn observed that 
always recording the 'improve English' 
goal doesn’t provide valuable feedback for 
either the learner or the program. 

Rethinking Goal Setting and Outcome 
Assessment: What Progress since 
1996? 

In the introduction to Volume 9 of 
Adventures in Assessment , Alison 
Simmons wrote: 

" Teachers know how to measure progress , 
but get stuck trying to find a tool or 
reporting mechanism to put their data in 
that would satisfy all the different audi- 
ences. The question goes beyond, 'how 
will I measure the progress of my students 
so it is meaningful to them, informs 
instruction and the curriculum and satis- 
fied the funder *. 

“Rather the questions should be ’ how do 
I report the progress / have seen to satis- 
fy different audiences? How do I report the 
progress my students have seen [in them- 
selves]? What information is useful to me 
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"Many feel that 
these in-class and/or 
program-based 
formative processes 
are in conflict with 
the state mandate 
for summative goal 
assessment.” 
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in my program , and what information do 
I need to satisfy other audiences?” 

Alison referred to the many forms of 
alternative assessment she had seen 
developed over the years by 
Massachusetts practitioners, including 
checklists, portfolios, and benchmarks, as 
well as the standardized tools. She also 
forecast the likely requirement for more 
systematic and standardized assessment 
to satisfy both public and private funding 
sources. For this reason, she concludes: 

" . . we need to validate and encour- 
age teachers to continue to develop sys- 
tems that inform instruction and show 
students where they are making gains , 
and how far they have come in achieving 
their goals.” 

Through my limited research, I have 
learned that Massachusetts practitioners 
do continue to develop and utilize creative 
tools and methods for formative assess- 
ment, and classroom processes for identi- 
fying, evaluating and revising/expanding 
learning goals. As noted, many feel that 
these in-class and/or program-based form- 
ative processes are in conflict with the 
state mandate for summative goal assess- 
ment. 

The Position at the Top: Process Can 
Complement Outcome Data Collection 

In a recent exchange on the National 
Literacy Advocacy listserv, Massachusetts 
Director of Adult Basic Education Robert 
Bickerton wrote the following: 

"For several years in Massachusetts , we 
have made the recording of student artic- 
ulated goals for their participation in ABE 
the centerpiece of what we mean by ' per- 
formance accountability ' - predating the 



Workforce Investment Act [and the NRS]. 
The policy is to record whatever the stu- 
dent says is the reason or reasons why 
she/he has decided to enroll in the pro- 
gram and to update these goals as the 
students' skills , abilities , dreams , aspira- 
tions and perspectives evolve and, possi- 
bly, solidify. 

“Not only have programs been under NO 
pressure from the state to curtail how 
ambitious and multiyear many of these 
goals are, but our office has often been in 
the position of encouraging and counsel- 
ing programs to not curtail what students 
articulate at their end. High expectations 
are embraced by everyone - in theory. But 
are hard for some to accept in practice. 
With WIA came the expectation that the 
student's ' primary goal ' would be achiev- 
able within a single program year. 

“I and others in our state strongly object- 
ed to this requirement arguing that it 
would pervert having student-articulated 
goals . . . and could foster ' creaming . ' For 
a while it appeared as if this would 
become a sticking point in our negotia- 
tions with the US DOE and we were pre- 
pared for a battle. Instead we found a way 
to tweak the existing policy. WE still honor 
the goals articulated by each student 
regardless of how long it may take to 
achieve them. We now also ask programs 
to discuss with each student what goals 
(or benchmarks) he or she would like to 
achieve in the program year.” 

ACLS Director Bickerton wrote, 
“Massachusetts DOE, including its profes- 
sional development arm, the SABES sys- 
tem, needs to work with the field to devel- 
op benchmarks and standards for learning 
gains and goal achievement that are 
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responsive to the needs and circumstances 
of the very diverse students who come to 
us for services 

There are rich resources and knowledge 
of ABE learner purposes and goals within 
the community of practitioners and learn- 
ers throughout the ABE system in 
Massachusetts. Creative practitioners resist 
the pressure to make 'goal setting' into an 
exercise isolated from the teaching/learn- 
ing process. Dynamic practitioners such as 
those quoted in this article want to 
demonstrate accountability for their work 
with learners. The challenge to ABE lies in 
promoting and sharing the many authentic 
models and practices for identifying learn- 
er purposes and goals in their own terms, 
and at the same time providing outcome 
data for learner achievement that will con- 
firm the 'performance accountability 1 that 
is alive and well in ABE classrooms across 
the state. 



Sally Gabb is the SABES Regional 
Coordinator for Southeastern 
Massachusetts. 
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Qu insigamond Community College's Site-Speci 



The new site-specific ESL assess- 

t ment test that follows was 
developed in response to our 
instructors’ concerns about the 
accuracy and scoring of our pre- 
vious site-specific assessment test. The 
original test didn’t cover the placement or 
assessment of the SPL 7-10 level students. 
With the addition of four higher-level ESL 
classes, instructors and Learning Support 
Specialists needed to be able to accurate- 
ly assess and place students who fell into 
that category. 

There was also a need for a test that 
was more objective and more standard- 
ized in its scoring. The older assessment, 
with its informal scoring, was adequate for 
assessing and placing students at the SPL 
0-2, 3-4, and 5-6 levels, but the instruc- 
tors began asking for more specific infor- 
mation about their student’s oral, reading, 
writing, and grammar skills. These con- 
cerns and issues prompted our ESL 
instructors to create the new test. 

Anne Burke, ESL Instructor, SPL 0-2 
(beginner), Linda Gosselin, ESL Instructor, 
SPL 5-6 (intermediate), and Arpi 
Hedeshian, ESL Instructor, 7-10 (advanced) 
formed a committee to design a new 
assessment test. They all agreed that it 
was time for a new test and the contents 
of the old test needed to be upgraded. 
They held several meetings during the 
summer. They looked over samples of 
tests being used in the ABE program at 
QCC and standardized and alternative 
tests being used by other programs. They 



used the standardized BEST Test as a 
guide to create their new test. They want- 
ed to make sure their new test covered all 
the skill areas. They also surveyed all the 
ESL instructors and Learning Support 
Specialists to ask if the current test placed 
their students accurately and if the scoring 
gave enough information about their stu- 
dents’ oral, reading, writing, and grammar 
skills. They asked for suggestions to revise 
the current assessment test. 

The new test still assesses the four skill 
areas (oral, grammar, reading, writing), but 
the administering and scoring of the test 
provides more accurate placement and 
scoring. 

The new assessment test was piloted 
during the summer of 2000 and is still 
being piloted throughout the 2000-2001 
school year. The initial committee will 
review and revise the test this spring if 
necessary. 



Christina Hebert is the Program Manager 
for the Quinsigamond Community 
College's Adult Community Learning 
Center. Anne Burke has been teaching the 
beginning ESOL Level at QCC's ACLC for 
the past ten years . Linda Gosselin has 
been teaching the intermediate ESOL 
level at QCC's ACLC for the past fifteen 
years. Arpi Hedeshian is now a Learning 
Support Specialist at three of QCC ACLC's 
sites. 
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Quinsigamond Community College Adult Basic Education Program 
ESOL INITIAL ASSESSMENT 

Directions for Administration and Scoring of the Test of Aural/Oral Skills 
and Test of Grammar, Reading, and Writing 



Applicant's Name 



Date 



General Information on Testing and Scoring 



Testing: 

There are three tests: The Test of Aural/Oral Skills, The Literacy Test, and the Test of Grammar, Reading, and 
Writing. The Aural/Oral Test should be administered first because the score from this test determines whether an 
applicant will take the Literacy Test(for low level applicants) or do the Test of Grammar, Reading, and writing (for 
those who are beginning to advanced level). 

Aural/Oral Test - 

This test consists of three parts: an oral interview, looking at a picture and answering questions about it, and a 
dictation. This test is administered to ail ESOL applicants, with testing ending when it is obviously too difficult for 
the applicant. More detailed instructions are found in the directions for the aural/oral test. 

Literacy Test - 

This test is given to any applicant who scores less than 10 points on the aural/oral test or is unable to do the 
aural/oral test at all. This test consists of two parts: dictating letters and numbers to the applicant and having the 
applicant match words and pictures. More information on the administration of the literacy test is in this direction 
booklet. 

Test of Grammar, Reading, and Writing - 

This test is done by all applicants who were able to do all of the aural/oral test or get more than 10 points on 
the aural/oral test. The test booklet is given to the applicant and the applicant works on his or her own to complete 
the test. The applicant may not use a dictionary or be assisted by anyone with the test, other than to clarify the 
directions for each part of the test. 
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Scoring: 

Scoring for the sections of each test is included in the instructions in this booklet. After all testing is finished and 
each section of the tests the applicant has taken is scored, a cover score sheet should be completed, listing the SPL 
levels for the Aural/Oral Test, the Literacy Test and/or Grammar, Reading, and Writing. Then the applicant is assigned 
a placement level. The two tests the applicant has taken should be clipped together with the cover sheet so that 
staff can see an applicant's SPL levels in various skill areas at a glance. 

Aural/Oral Test 

To facilitate administering of this test and to ensure that different testers administer the test in the same manner, 
the directions for Parts A and B are at the beginning of each part. There are directions for administering Part C, the 
dictation, on the sheet containing the dictation sentences. 

Part A: Oral Interview 



There are ten questions in the oral interview. Ask each question and score the applicant's response according to 
the key shown at the beginning of the oral interview. When conducting the oral interview, if an applicant has a lot 
of trouble comprehending the first four questions, STOP THE INTERVIEW and assume a score of o for the rest of the 
interview questions. 

Do Parts B and C with applicants who score 10 or more points in the oral interview. If the applicant scores less 
than 10 points, don't have the applicant do Parts B and C. The interviewer will then give the literacy test to the appli- 
cant. At the end of the interview add the scores for each question and record the total in the space provided at the 
end of the oral interview part. 

Part B: Picture and Questions 



Using the picture card of the scene at a train station, show the picture to the applicant and ask the six questions 
about the picture. Be sure to follow the instructions to the tester that appear in parentheses for each question. 
Scoring is the same as for Part A. At the end of Part B, add and record the total in the space provided. 

Part C: Dictation 



The directions for administering the dictation are also provided on the dictation sheet. Select one of the dictation 
sentences from each level (for a total of three sentences) to dictate to the applicant. If it is obvious that the appli- 
cant can't do the dictation, stop after attempting the first sentence. If the applicant attempts the dictation but has 
great difficulty, don't have him/her try an advanced sentence. 

When reading each sentence, pause after each slash mark to give the applicant time to write one phrase before 
going on to the next phrase. If the applicant asks, a phrase can be repeated one time. 

The scoring for the dictation is at the bottom of the dictation page in the test. Circle the total for how the appli- 
cant did in doing the dictation. 
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ESOL initial Assessment 
Part C: Dictation 

ADMINISTRATION: 

Select one sentence from each level to dictate to the applicant, if it is obvious that the applicant can't do the 
dictation, stop at the first sentence. If the applicant attempts the dictation, but has great difficulty, don't have 
him/her try the advanced sentences. 

DICTATION SENTENCES: 

Beginning Level: 

1. I go / to school. 

2. Ten cats sit / in the tree. 

Intermediate Level: 

1. The children ran /into the house. 

2. He is taking / his vacation in August. 

Advanced Level: 

1. My neighbor was stopped / by the police for drunk driving, 

2. The leaves change color / and fall from the trees in autumn. 
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SCORING FOR THE AURAL/ORAL TEST 

Add the scores for the three parts of the test and record the score in the Grand Total space at the bottom of the 
dictation page. The total numbers of points for this test are 100. Record the applicant's oral SPL level in the space 
provided on the front cover of the Aural/Oral Test. The breakdowns of scores by SPL level are as follows: 



90-100 points 


= 


SPL 10 


80-89 points 


= 


SPL 9 


70-79 points 


= 


SPL 8 


60-69 points 


= 


SPL 7 


50-59 points 


= 


SPL 6 


40-49 points 


= 


SPL 5 


30-39 points 


= 


SPL 4 


20-29 points 


= 


SPL 3 


10-19 points 


= 


SPL 2 


1-9 points 


= 


SPL 1 


do/understand anything 


= 


SPL 0 






LITERACY TEST 



Administration 

Give the literacy test to any applicant who cannot do the oral interview or who scores less than 10 points on the 
oral interview. 

Part A: Dictation 



Dictate the following random order letters and numbers to the applicant and have him/her write them on the lines 
provided in the literacy test. See the dictation card for the letters and numbers to be dictated. 

Letters: tresbi mgyal dop 

Numbers: 7 3 6 1 2 11 18 20 45 99 100 

Part B: Reading (Matching Pictures and Words) 



Have the applicant do Part B on his/her own. DO NOT READ THE WORDS TO THE APPLICANT. The applicant is to 
look at each word or sentence and put a circle around the picture that illustrates the word or sentence. 
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Scoring: 



Part A: Dictation 

Count the number of letters and numbers the applicant got correct and score as follows: 



12 or less correct 


= SPL 0 


13-24 correct = 

25 correct(no errors) 


SPL 1 

= SPL 2 


Part B: Reading(Matching) 

Count the number of matchings the applicant got correct and score as follows: 


4 or less correct 


= SPL 0 


5-9 correct 


= SPL 1 


10 correct 


= SPL 2 



GRAMMAR, READING, AND WRITING TEST 



Administration 

If the applicant was able to do all sections of the aural/oral test and did not need to do the literacy test, give 
the applicant the test booklet for the Grammar, Reading, and Writing Test and have him/her do the test. After the 
applicant has finished the test, check to be sure he/she has tried to do all sections of the test and didn't skip a 
page or section by accident. 

Scoring 

After checking each section, record the scores on the score sheet. 

Part A: Grammar 22 Sentences 



Count the number of items the applicant got correct and record the score as follows: 



0-4 correct 


= 


SPL1-2 


5-9 correct 


= 


SPL 3-4 


10-14 correct 




SPL 5-6 


15-19 correct 




SPL 7-8 


20-22 correct 


= 


SPL 9-10 
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Part B: Reading 7 Questions 



Add the number of correct responses. 



Record the score as follows: 




None correct 


SPL o-2 


1-2 correct = 


SPL 3-4 


3-4 correct 


SPL 5-6 


5-6 correct 


SPL 7-8 


7 correct 


SPL 9-10 



Part C: Writing about a Picture 

Look at the applicant's writing and record the score as follows: 

No writing = SPL 0-2 

Writes isolated words & phrases, with many grammar, spelling, and 
syntax errors = SPL 3-4 

Writes short, simple sentences with some grammar, spelling, and syntax 
errors = SPL 5-6 

Writes more complex sentences with fewer grammar, spelling, and syntax 
errors = SPL 7-8 

Writes very well with very few grammar, spelling, or syntax 
Errors = SPL 9-10 
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ESOL INITIAL ASSESSMENT 

Quinsigamond Community College Adult Basic Education Program 



TEST OF AURAL/ORAL SKILLS 



Applicant's Name Date 

Oral SPL Level Interviewer 
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